Multi-modal and multi-hop question answering aims to answer a question based on multiple input sources from different modalities. Previous methods retrieve the evidence separately and feed the retrieved evidence to a language model to generate the corresponding answer. However, these methods fail to build connections between candidates and thus cannot model the inter-dependent relation during retrieval. Moreover, the reasoning process over multi-modality candidates can be unbalanced without building alignments between different modalities. To address this limitation, we propose a Structured Knowledge and Unified Retrieval Generation based method (SKURG). We align the sources from different modalities via the shared entities and map them into a shared semantic space via structured knowledge. Then, we utilize a unified retrieval-generation decoder to integrate intermediate retrieval results for answer generation and adaptively determine the number of retrieval steps. We perform experiments on two multi-modal and multi-hop datasets: WebQA and MultimodalQA. The results demonstrate that SKURG achieves state-of-the-art performance on both retrieval and answer generation.

Calibration strengthens the trustworthiness of black-box models by producing better accurate confidence estimates on given examples. However, little is known about if model explanations can help confidence calibration. Intuitively, humans look at important features attributions and decide whether the model is trustworthy. Similarly, the explanations can tell us when the model may or may not know. Inspired by this, we propose a method named CME that leverages model explanations to make the model less confident with non-inductive attributions. The idea is that when the model is not highly confident, it is difficult to identify strong indications of any class, and the tokens accordingly do not have high attribution scores for any class and vice versa. We conduct extensive experiments on six datasets with two popular pre-trained language models in the in-domain and out-of-domain settings. The results show that CME improves calibration performance in all settings. The expected calibration errors are further reduced when combined with temperature scaling. Our findings highlight that model explanations can help calibrate posterior estimates.

Pre-trained Language Models (PLMs) have been applied in NLP tasks and achieve promising results. Nevertheless, the fine-tuning procedure needs labeled data of the target domain, making it difficult to learn in low-resource and non-trivial labeled scenarios. To address these challenges, we propose Prompt-based Text Entailment (PTE) for low-resource named entity recognition, which better leverages knowledge in the PLMs. We first reformulate named entity recognition as the text entailment task. The original sentence with entity type-specific prompts is fed into PLMs to get entailment scores for each candidate. The entity type with the top score is then selected as final label. Then, we inject tagging labels into prompts and treat words as basic units instead of n-gram spans to reduce time complexity in generating candidates by n-grams enumeration. Experimental results demonstrate that the proposed method PTE achieves competitive performance on the CoNLL03 dataset, and better than fine-tuned counterparts on the MIT Movie and Few-NERD dataset in low-resource settings.

现代实体链接（EL）系统构成了流行偏见，但是没有数据集以英语以外的其他语言上关注尾巴和新兴实体。我们向Hansel展示了中国人的新基准，它填补了非英国几乎没有射击和零击EL挑战的空缺。Hansel的测试集经过人工注释和审查，并采用了一种用于收集零照片EL数据集的新方法。它涵盖了新闻，社交媒体帖子和其他网络文章中的10k多种文档，Wikidata作为目标知识库。我们证明，现有的最新EL系统在Hansel上的表现不佳（R@1中的36.6％，几乎没有射击）。然后，我们建立了一个强大的基线，该基线在我们的数据集上的零射门上为46.2％的R@1分之1。我们还表明，我们的基线在TAC-KBP2015中国实体链接任务上取得了竞争成果。

与自然语言解释的视觉结合旨在推断文本图像对之间的关​​系并生成句子以解释决策过程。先前的方法主要依靠预先训练的视觉模型来执行关系推断和语言模型来生成相应的解释。但是，预训练的视觉模型主要在文本和图像之间建立令牌级别的对齐，但忽略了短语（块）和视觉内容之间的高级语义对齐，这对于视觉推理至关重要。此外，仅基于编码的联合表示形式的解释生成器并未明确考虑关键的关系推理的决策点。因此，产生的解释不太忠于视觉语言推理。为了减轻这些问题，我们提出了一种统一的块意见对齐和基于词汇约束的方法，称为CALEC。它包含一个块感知的语义交互器（ARR。CSI），一个关系属性和词汇约束感知的发生器（arr。Lecg）。具体而言，CSI利用语言和各个图像区域固有的句子结构来构建块感知语义对齐。关系下属使用基于注意力的推理网络来合并令牌级别和块级视觉语言表示。 LECG利用词汇约束来将关系下列者重点关注的单词或块纳入解释世代，从而提高了解释的忠诚和信息性。我们在三个数据集上进行了广泛的实验，实验结果表明，CALEC在推理准确性和生成的解释的质量方面显着优于其他竞争者模型。

对话系统已取得了重大进展，并已在各种情况下广泛使用。先前的研究主要集中在单个情况下设计对话模型，而在现实世界中各种情况下处理任务需要全面的能力。在本文中，我们提出了一个通用的多技能对话框框架，即MSDF，可以应用于不同的对话框任务（例如，知识接地对话框和基于角色的对话框）。具体而言，我们提出了一个可转移的响应生成器，以在多种大规模对话库中进行预训练，作为MSDF的骨干，由基于BERT的编码器和基于GPT的解码器组成。为了选择与对话记录一致的响应，我们提出了一个通过负抽样训练的一致性选择器。此外，还采用了外部知识的灵活复制机制来增强各种情况下多形知识的利用。我们对知识接地对话，建议对话框和基于角色的对话任务进行实验。实验结果表明，我们的MSDF的表现优于基线模型。在2021年语言和情报挑战的多技能对话中，我们的一般MSDF赢得了第三奖，这证明我们的MSDF具有有效且具有竞争力。

医学对话生成是一项重要但具有挑战性的任务。以前的大多数作品都依赖于注意力机制和大规模预处理的语言模型。但是，这些方法通常无法从长时间的对话历史中获取关键信息，从而产生准确和信息丰富的响应，因为医疗实体通常散布在多种话语中以及它们之间的复杂关系。为了减轻此问题，我们提出了一个具有关键信息召回（Medpir）的医疗响应生成模型，该模型建立在两个组件上，即知识吸引的对话图形编码器和召回增强的生成器。知识吸引的对话图编码器通过利用话语中的实体之间的知识关系，并使用图形注意力网络对话图来构建对话图。然后，召回增强的发电机通过在产生实际响应之前生成对话的摘要来增强这些关键信息的使用。两个大型医学对话数据集的实验结果表明，Medpir在BLEU分数和医疗实体F1度量中的表现优于强大的基准。

最近的作品表明了解释性和鲁棒性是值得信赖和可靠的文本分类的两个关键成分。然而，以前的作品通常是解决了两个方面的一个：i）如何提取准确的理由，以便在有利于预测的同时解释; ii）如何使预测模型对不同类型的对抗性攻击稳健。直观地，一种产生有用的解释的模型应该对对抗性攻击更加强大，因为我们无法信任输出解释的模型，而是在小扰动下改变其预测。为此，我们提出了一个名为-BMC的联合分类和理由提取模型。它包括两个关键机制：混合的对手训练（AT）旨在在离散和嵌入空间中使用各种扰动，以改善模型的鲁棒性，边界匹配约束（BMC）有助于利用边界信息的引导来定位理由。基准数据集的性能表明，所提出的AT-BMC优于分类和基本原子的基础，由大边距提取。鲁棒性分析表明，建议的AT-BMC将攻击成功率降低了高达69％。经验结果表明，强大的模型与更好的解释之间存在连接。

Masked image modeling (MIM) performs strongly in pre-training large vision Transformers (ViTs). However, small models that are critical for real-world applications cannot or only marginally benefit from this pre-training approach. In this paper, we explore distillation techniques to transfer the success of large MIM-based pre-trained models to smaller ones. We systematically study different options in the distillation framework, including distilling targets, losses, input, network regularization, sequential distillation, etc, revealing that: 1) Distilling token relations is more effective than CLS token- and feature-based distillation; 2) An intermediate layer of the teacher network as target perform better than that using the last layer when the depth of the student mismatches that of the teacher; 3) Weak regularization is preferred; etc. With these findings, we achieve significant fine-tuning accuracy improvements over the scratch MIM pre-training on ImageNet-1K classification, using all the ViT-Tiny, ViT-Small, and ViT-base models, with +4.2%/+2.4%/+1.4% gains, respectively. Our TinyMIM model of base size achieves 52.2 mIoU in AE20K semantic segmentation, which is +4.1 higher than the MAE baseline. Our TinyMIM model of tiny size achieves 79.6% top-1 accuracy on ImageNet-1K image classification, which sets a new record for small vision models of the same size and computation budget. This strong performance suggests an alternative way for developing small vision Transformer models, that is, by exploring better training methods rather than introducing inductive biases into architectures as in most previous works. Code is available at https://github.com/OliverRensu/TinyMIM.

The recent increase in public and academic interest in preserving biodiversity has led to the growth of the field of conservation technology. This field involves designing and constructing tools that utilize technology to aid in the conservation of wildlife. In this article, we will use case studies to demonstrate the importance of designing conservation tools with human-wildlife interaction in mind and provide a framework for creating successful tools. These case studies include a range of complexities, from simple cat collars to machine learning and game theory methodologies. Our goal is to introduce and inform current and future researchers in the field of conservation technology and provide references for educating the next generation of conservation technologists. Conservation technology not only has the potential to benefit biodiversity but also has broader impacts on fields such as sustainability and environmental protection. By using innovative technologies to address conservation challenges, we can find more effective and efficient solutions to protect and preserve our planet's resources.